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ABSTRACT 

As a dimension of a total school reform initiative, the Delcroft School 
faculty, committed to a more integrative literacy curriculum, implemented 
a proposed framework to align teaching/learning/assessment. This report 
documents the qualitative process and quantitative outcomes of the effort. 
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THE FRAMEWORK 

In March 1993, Rhonda Lauer, Superintendent of the Southeast Delco 
Schools in Pe nn sylvania, asked Morton Bo tel to join the staff in developing, 
implementing and researching their Goals 2000 reform plan, fulfilling the 
requirement of all Pennsylvania school districts to meet the Curriculum 
Standards of the Commonwealth. 

Lauer had just obtained a provisional award of $250,000 per year for 
five years from the ARCO Corporation to enable the schools to develop and 
implement an integrative and comprehensive school improvement plan. In 
particular, she was interested in having him design and research a 
comprehensive framework that would link teaching/ leaming/assessment. 
One of the requirements of the assessment framework was that it would 
yield quantitative as well as qualitative evidence of student outcomes, 
because we wanted several kinds of evidence on how well the reform effort 
was working. Also, as a major stakeholder in the success of the reform, 
ARCO wanted quantitative evidence of the effectiveness of the reform plan 
at the end of each funding year as a key basis for determining whether or 
not to continue their financial support. As it turned out, ARCO backed out 
of its support at the end of two years because Superintendent Lauer, who 
got the grant, left the district. But the staff of Delcroft School determined 
that they would continue the process and we agreed to continue working 
with them. 

Specifically, Botel proposed a framework that would enable the 
Delcroft teachers to work through a dynamic process to develop a more 
authentic curriculum-sensitive assessment system using written 
retellings. This process would incorporate articulated learning 
experiences: talking/writing/reading across the curriculum. 

At the time we began, Bonnie George, the principal of the Delcroft 
school, a K-5 school in a working class community, and her faculty 
volunteered to pilot the project. The Delcroft staff was already committed 
to implementing the integrative curriculum framework of the Pennsylvania 
Department of Education: THE PENNSYLVANIA FRAMEWORK: Reading, 
Writing and Talking across the Curriculum (Lytle & Botel,1988). Many of 
the teachers at Delcroft School had completed or were taking a one or two 
year-long continuing education seminar based on THE PENNSYLVANIA 
FRAMEWORK, on site, offered by the PENN Literacy Network (Botel, Ripley 
& Barnes, 1992, Botel, Botel-Sheppard & Renninger, 1994). 

Botel's proposal was influenced by the conclusions of Peter 
Drucker(1985). Drucker investigated a number of fields to determine the 
bases on which innovations become successful since he was struck with the 
fact that historically very few innovations endure. Drucker concluded 
from his research that if an innovation was to survive, two principal 
conditions must be met: 1) t£e innovation needed to be reasonably simple 
and yet have a comprehensive system for implementation, dissemination 
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and accountability and 2) the effort had to begin small. Keeping these 
concepts in mind, the Botel proposal, following consultation with Delcroft 
teachers on various options for a written retelling performance assessment, 
began with the following five framing ideas: 

1. The pre and post written retelling activities each year from grades 
K to 5 would include both literary and expository texts. Children would be 
assessed on the analysis of written retellings of each of these texts for 
comprehension of content and separately for mechanics. Using both 
literary and expository texts would symbolically represent a commitment 
to improve teaching/learning across the curriculum. 

2. The written retellings would be assessed by all teachers working 
collaboratively in grade level groups. Assessment would be both 
qualitative and quantitative. The qualitative aspect of the assessment 
would involve two dimensions: 1) the teachers' insights and judgment in 
the dynamic process of discussing children's written retellings with their 
colleagues, 2) teachers' perception of the effect of their collegial discussions 
on teaching/learning in their classrooms over the course of the project, and 
3) observations of the authors of this report. This qualitative evidence 
along with the quantitative assessment of the pre and post written retelling 
activities would inform all stakeholders of the process and outcomes of the 
program. 

3. Teachers over the five years would continue to develop and use 
written retellings, many other reading/writing/talking activities and other 
productive learning experiences such as discussions, dramatizations, and 
illustrations to enhance children's comprehension and construction of 
meaning across the curriculum. 

4. The entire process would involve continuous opportunities for 
professional development for the entire staff, practically of it to be held 
during the regular school day over the course of the study. 

5. In support of the district's newly instituted "full inclusion" policy, 
all teachers and all children in the school would be involved in the 
teaching/learning/assessment activities. 

Underlying these principles is the unifying and enabling idea 
expressed by Darling-Hammond regarding a successful reform process. It 
"...should aim to create a system in which improved teacher knowledge and 
equalized school capacity are the starting points for systemic change. In 
such a system, teachers and schools will have the knowledge, resources, 
and organizational supports to create appropriate curriculum and useful 
assessments for the students they serve (1994, p. 478). 

WHY WRITTEN RETELLINGS? 

There are several reasons why a written retelling process as we 
conceived it was appropriate First, it was an integrative teaching/learning 
activity that involved both literary and informational content which 
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children experienced through listening, talking, writing and reading. The 
written products, in addition to having been a consequence of a holistic 
learning experience was a performance that could be assessed 
quantitatively. 

Second, the research findings on story retellings suggest that they 
enhance oral language, reading comprehension and knowledge of story 
structure (Keefe, 1992; Gambrell, Koskinen & Kapinus 1991; Golden and 
Pappas, 1987; Goodman, Watson & Burke, 1987; Morrow, 1986, 1992). In 
Morrow's (1992) review of story retelling research, she made the connection 
between teaching and assessment: "Story retelling offers not only an 
instructional technique, but an evaluative one as well... Because it is a 
holistic procedure, retelling is strikingly different from the traditional 
piecemeal questioning approach to developing and assessing 
comprehension." (p.50). In fact the written retelling assessment might 
reasonably be called a "comprehensive literacy assessment." 

Third, by having the teacher read the texts aloud first, all children 
will hear good interpretations of the texts and thus be enabled to be 
successful in experiencing the texts. If children were required to read the 
text first, kindergartners and first graders and many slow readers at the 
higher grade levels would have found the texts to be at their frustration 
levels and therefore incomprehensible. 

Fourth, by having children in pairs retell the narrative and 
informational texts to one another after hearing them, all will get an 
opportunity to collaborate with peers by orally rehearsing the 
reconstruction of the story before they write down their versions of the 
stoiy. 

Fourth, by having children reflect on and revise their written 
retellings over several days, they get valuable experience with the writing 
process. 

Finally, a written retelling for each child, makes it easier and less 
time consuming for teachers to assess the children's retellings as compared 
with the more common practice of assessing each individual child's oral 
retelling of a text they have read. 

Teachers at Delcroft were given a how to do it framework based on 
the principles stated above. The current version of the framework, which 
appears in the appendix of this report, benefits from our experience in 
implementing the framework and therefore is useful for those who would 
initiate a similar process. Throughout the professional development 
process teachers were constantly encouraged to apply their considerable 
knowledge of teaching and of their children to make the process work on 
their behalf. 
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RESEARCH QUESTIONS 

In the research dimension of this reform effort we wanted to find 
answers to two major questions: 

1. How and how well was the written retelling reform implemented? 

2. Did the children, on the average, make significant "value added" 
improvement in their written retellings in content and mechanics on both 
narrative and expository texts? 

THE IMPLEMENTATION AND PROFESSIONAL DEVELOPMENT 

PROCESS 

Introduction 



(While the Appendix details the current Botel framework, we indicate 
here briefly the features of the proposal so as to make sense of the rest of 
this report. First, the teacher reads the text aloud twice. Then children in 
pairs retell the story or informational text to one another. Each child then 
does a written retelling draft. For two more days they work on revision of 
their drafts. K-l teachers score each written retelling on a revised Marie 
Clay 8-point scale. Second to fifth grade teachers, meeting in grade level 
groups, rank all written retellings at their grade level, develop the local 
norms and score all retellings based on them. These norms are used for the 
five years of the study.) 



That first year was busy! In late May of 1993, we introduced the 
Delcroft faculty to what we had begun to call "The Written Retelling Project 
through a ninety minute "mini-workshop." The most enjoyable and 
probably the most convincing part of this presentation involved Conne 
leading her colleagues through a retelling of James Thurber's modem 
folktale "The Scotty Who Knew too Much" (a parable related to typical, 
"bring in the outside expert" approaches to staff development!). After 
listening to the story read aloud, colleagues paired up and told the story to 
each other. Although they did not have time to sample the revising and 
editing activities, teachers had a chance to write first drafts. Before the 
workshop, w r e worried about teachers' responses to the proposal; we knew r 
it would not succeed if they did not believe in its worth. When two teachers 
shared their very different but equally humorous and well-written 
versions of "The Scotty" to appreciative applause, we knew we were on our 
way . 

Almost every teacher (all but one) volunteered to try out a narrative 
retelling process in her classroom before the end of the year. Asked to 
identify questions and concerns arising from this experience, teachers 
wondered, "How much staff development will be provided?' "Will time be 
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provided for teachers to meet?" and "How will our report card change to 
reflect this process?" One teacher commented that, in order to rate the 
retellings, "we have to be able to determine what we want students to be 
able to do." 

Despite these concerns, it was already apparent that many teachers 
were excited about the project and shared our hopes for some of its 
benefits. They said, "This is great for involving more teachers in writing." 
"Retelling is a comfortable start for those uncomfortable with writing." 

"The team approach is encouraging," and "This allows for differences in the 
learning styles of our children." 

In June, Delcroft's principal Bonnie George and Conne met with each 
grade level team to determine times during the summer when they could 
participate in a workshop that would take place across three half days. 
Largely due to Bonnie’s demonstrated support for the project, eighty 
percent of Delcroft's faculty signed up for one of the two workshop series. 
Because of the Arco Corporation grant, the school district was able to pay 
teachers for the work they would be doing during these sessions. 

We had five goals for the summer workshops— goals which reflect our 
emphasis on teachers' active participation in the adaptation (rather than 
the adoption) of the original proposal. The original "Workshop Agenda" 
lists these objectives: 1) Refine beginning and end of year story/news 
retelling process; 2) Create list of ideas for integrative language activities 
encouraging students to take on roles of professional readers and writers; 

3) Introduce possibilities for qualitative assessment; 4) Practice and refine 
the rating process: and, 5) Create a plan for continuing professional 
development. Clearly, we hoped that teachers would not only adapt the 
retelling and rating processes to work in their classrooms, but that they 
would also generate ideas for related teaching/learning/assessment 
activities and plan for their own and their colleagues' ongoing professional 
development. 

During the workshop sessions, teachers met in small groups— often 
with grade level partners-tried out an expository retelling, planned 
sample lessons, brainstormed issues and solutions, practiced rating student 
papers (collected during the spring 'try out'), looked closely at samples of 
student writing using a descriptive review process , and created specific 
guidelines for retelling and rating processes at each grade level. At times, 
the process of creating "standardized" grade level procedures was 
harrowing, making decisions about the retelling and rating processes raised 
many important but thorny questions about teaching, learning and 
assessment. These issues ranged from specific questions about the process- 
-"Should we show the story on an overhead while we read it aloud?" "Can 
students draw instead of taking notes?" "How do we choose revision 
partners?"- to more general questions about writing and assessment - "Does 
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retelling assess reading or writing or both?" "How important is creativity?" 
"What makes an expository retelling really 'good'?" 

What became clear during these three half days was the energy and 
the professional knowledge that teachers brought to the process. Given the 
opportunity to make decisions about how the assessment should be 
conducted, they took this responsibility seriously and worked hard to adapt 
the original proposal to fit their contexts. This work necessarily involved 
difficult compromises; since beginning and end of year retelling and rating 
processes needed to be "standardized" by the teachers for each grade level, 
there were times when one teacher might have to accept a decision that did 
not feel right to her. Fortunately, teachers knew that the beginning and 
end of the year retellings were only the beginning, and that, throughout the 
year, they could try out many variations in their classrooms. 

Adaptations to Original Proposal 

Since the beginning and end of year retelling process, once 
established, needed to remain standardized for the five years of the project, 
we encouraged Delcroft faculty to make adaptations to the original retelling 
process specified in Botel's proposal. Some changes were made across 
grade levels; these included 1) developing some prior knowledge of the 
story topics with students before reading them the story; 2) either showing 
students the story itself or using some type of visual aid during the initial 
story reading by the teacher; and 3) encouraging students to add to their 
notes after the second reading of the story. Other changes were grade 
specific. For example, because they believed that part of what they wanted 
to assess was their students’ ability to use appropriate resources when 
writing, teachers in grade levels two and above decided to allow students to 
use dictionaries and thesauruses during the editing process if they 
requested them. Kindergarten and Grade One teachers felt that their 
students were not developmen tally ready to benefit from Day Three's 
editing process, and revised the retelling procedure to include only two 
days. Each grade level separately refined the revision and editing 
checklists provided in the original proposal so that their language was 
appropriate; in second grade, for instance, the editing checklist included 
actual punctuation marks along with their labels. All of these changes were 
accompanied by a great deal of discussion and negotiation. Although it was 
sometimes arduous, ultimately many teachers found this process useful and 
commented on the pleasure of hearing and learning from knowledgeable 
colleagues. Some issues were never "settled" to a grade level groups' 
satisfaction, and this has led to continuing conversations about issues such 
as how to pair up students for revision, how long to allow students to draw 
rather than take notes, and how to introduce the retelling process to 
students in a way that encourages them to be creative as well as to 
demonstrate a grasp of the "j>ist" of the story in their retelling. 
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During the summer workshops, while doing "practice" ratings with 
actual student papers, Delcroft teachers adapted the rating process, too. In 
kindergarten and first grades, the major adaptation was to add the phrase 
"related to the story" to levels 4 through 8 of Marie Clay’s rating scale (thus, 
level four becomes "Any recognizable word related to the story.") This 
change allowed students’ ratings to reflect their comprehension of the 
story, as well as the development of their facility with writing. Because of 
their excitement with their students' work, Kindergarten and Grade One 
teachers also decided to begin developing, for future use, another language 
development scale focused on meaning and verbal fluency which they 
would use to describe and assess story retellings that students dictated 
rather than wrote out themselves. For grades Two and above, teachers 
agreed to stick with the rating process established in the original proposal; 
however, they refined them by establis hin g efficient procedures for rating 
both content and mechanics. One group recommended that each teacher 
(except the reader) "vote" on a certain rating by showing one, two, three or 
four fingers (similar to the Scissors, Rock, Paper game!) after a paper was 
read aloud; this easy process caught on and ended up being used at most 
grade levels. For rating mechanics, most groups passed papers to each 
teacher who would write a rating on the back of the paper. The most 
prevalent mechanics rating would be assigned to that piece. 

One of the most difficult, yet ultimately rewarding, aspects of rating 
sessions over the past three years has been developing a shared sense of 
what constitutes a good narrative or expository retelling. A question that 
has arisen again and again has been whether the ratings should reflect 
"comprehension" or "writing ability." Fortunately, this question has turned 
out to be largely academic; the most creative retellings have also 
demonstrated a good grasp of the point of the story, if not the specific 
details. In addition, although "what exactly we’re rating" has remained an 
issue for some teachers, others have expressed a growing appreciation for 
the validity of a holistic approach to rating which does not necessitate 
trying to separately assess a student's understanding of a story and his or 
her ability to communicate that understanding. This shift has been 
reflected in the rating process as teachers have begun to pay less attention 
to some specific criteria which we generated and listed after first choosing 
anchor papers for each rating level and paying more attention to the anchor 
papers themselves as benchmarks against which to rate new student 
papers. 

Continuing Professional Development 

At least one teacher from each grade level participated in the 
summer workshops; these teachers worked with us to educate their grade 
partners about the retelling and rating processes at a two hour all faculty 
meeting during fall orientatipn days. Over the past three years, other staff 
development sessions related to the project have included hour-long 
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workshops on Wednesday early release afternoons (three to five per year), 
many grade level team meetings devoted to talking about the retelling 
project and related activities, and two meetings where Delcroft faculty 
shared their concerns, questions and satisfactions with the authors of this 
report. However, according to Delcroft teachers, the most important 
professional development has taken place during the beginning and end of 
year rating sessions when grade groups have spent two mornings or 
afternoons (approximately three hour sessions) looking together at their 
students' writing. The value of this uninterrupted, focused professional 
time spent with colleagues can not be overrated, as we report more 
specifically later in this article. The benefits for teachers of these rating 
sessions ranged from increased knowledge of the writing abilities and 
needs of their students to heightened respect for the expertise of their 
colleagues. 

REPORT OF QUALITATIVE OUTCOMES 

The following report on the qualitative outcomes of this project is 
informed by a range of data collected by Cathy Luna and Conne Broderick 
over the past three years. After every meeting or rating session, we have 
asked Delcroft teachers to fill out a written reaction sheet, answering 
questions such as "What stood out for you from this meeting?" or "What 
questions or concerns do you have about the retelling project at this point?" 
Teachers have also answered several more general questionnaires and 
surveys about the impact of the retelling project, the most recent of which 
(Feb. 1996) we draw on extensively in this section. In addition, Cathy Luna 
has interviewed several Delcroft teachers, Delcroft's principal, Bonnie 
George, and two of the authors for this report, Morton Botel and Conne 
Broderick. Finally, along with site documents collected from meetings, 
rating sessions and classrooms, Cathy Lima's field notes from extensive 
participant observation over the past three years serve to inform our 
report on the effects of the Written Retelling Process at Delcroft. 

How Teachers Used Qualitative Information about Students' 
Reading/Writing/Talking Abilities 

Delcroft teachers report that participating in rating sessions has given 
them a great deal of information about the reading/writing/talking abilities 
and needs of their students, both as a group and as individuals. In 
particular, teachers cite story comprehension and the mechanics of writing 
as areas in which they have learned about their students during rating 
sessions. Teachers have also gleaned more process-oriented kinds of 
information about students from the actual retelling procedure. Watching 
students go through the three-day retelling process, teachers have been 
able to assess individual students’ abilities in listening, comprehension, note 
taking, oral expression, revision and editing. 
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In answer to the question "In what ways do you use information 
gained through the retelling and rating sessions?" (Questionnaire, February 
1996), most Delcroft teachers mentioned two ways in which they used 
what they learned: to guide whole class, small group or individual writing 
instruction, or to communicate with students and/or their parents about 
the student's strengths, needs and progress. As a guide to instructional 
planning, teachers across grade levels (K-5) have found the qualitative 
information gained from retelling and rating sessions to be invaluable. It 
has helped them to "pinpoint strengths and weaknesses," "target areas of 
concern," "plan mini-lessons on content and mechanics" and "determine 
groupings for small group instruction." 

As a tool for communicating with students and parents about 
progress and abilities, teachers of students in the lower grades have found 
the students' written retellings to be especially useful. One first grade 
teacher comments that "The students, parents and I can see the growth." 
The same teacher highlights the possibilities for encouraging reflection by 
youngsters when she reports, " The students are surprised at their own 
growth. They are beginning to look at their own papers and choose which 
one they think is best." (M. Wardynski, Questionnaire, 2/96). Teachers of 
older students have also used retellings as a springboard for student choice 
and reflection, often incorporating them into writing and/or reading 
portfolios containing both teacher and student chosen pieces as well as 
student reflections. In general, it seems that, in addition to guiding 
instruction and being a tool for communicating with others about student 
progress, the retellings also served to prompt teachers to involve students 
more directly in assessing their own progress by examining samples of 
their own work. 

Instructional Changes Related to the Written Retelling Project 

It's a common saying in education now that 'assessment drives 
instruction.' One of the primary purposes of the Written Retelling Process 
at Delcroft has been to provide an assessment that is congruent with and 
that clearly values the kinds of holistic literacy teaching practices that the 
teachers, the school, and the district have been attempting to move toward. 
As Delcroft's principal, Bonnie George, said during the project's pilot year, 
"One purpose of this assessment is to affect what is being taught - to drive 
the teaching. The hope is that this type of assessment will help us move 
away from testing (and teaching) for one word answers." (Interview 
2/15/94) Our research suggests that, to an exciting extent, the project is 
meeting this goal. Both our observations and teachers' reports indicate that 
taking students through a three-day written retelling process and then 
being involved in the collaborative assessment and rating of the students' 
written samples has made teachers keenly aware of the value of writing 
process approaches and has prompted them to emphasize writing to learn, 
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writing in the content areas, strategies for writing, active reading, learning 
to learn and collaborative learning activities. 

Perhaps least surprising has been a reported increase in teachers' use 
of actual retellings, both oral and written, in their classrooms; this makes 
sense as teachers have seen, first hand, that good retellings are valued. 
Teachers report asking students to do both oral and written retellings in 
response to literature and to content area readings. They also describe an 
interesting focus on drawing as part of retelling; some teachers have 
students draw in response to stories, illustrate science lessons, add details 
to drawings as a form of revision, sequence pictures or do oral retellings 
using illustrations as clues. 

In addition to more use of activities directly related to retellings, the 
project seems to have led to more writing, in general, in many classrooms. 
In answer to the question " What activities related to the written retelling 
have you used in your classroom, some teachers simply report "more 
writing," "daily writing," or, in the case of one kindergarten teacher, 
"introduced writing earlier in the year." The range of specific writing 
activities, in addition to retellings, that teachers cite include journal entries, 
ballads, letters, poems, books reports, summaries, time lines, movies, plays, 
travel brochures and advertisements. Delcroft students are being asked to 
use these types of activities across content areas. One explanation for this 
increase in writing activities could be that the retelling project helped some 
teachers begin to see writing in a new way. A fifth grade teacher explained 
that her involvement in the project resulted, for her in "a shift in 
paradigm"; she has begun to see writing as "a learning tool across the 
curriculum" (E. Maher, Questionnaire, 2/96). 

Another striking effect of the retelling project is an increased 
awareness of the importance of teaching students strategies - for writing, 
for reading and for learning to learn. Teachers report asking students to 
practice using prewriting techniques such as story mapping, revision 
activities such as peer conferencing, and editing activities such as using an 
editing checklist, and a dictionary or thesaurus. Active reading strategies 
that teachers use as a result of the retelling project include identifying 
story components, creating Venn diagrams and creating new texts using 
details gleaned from stories. Learning to learn strategies that some 
teachers did not realize their students needed until they watched them 
during Day One of the retelling procedure include note taking from listening 
and from reading, outlining, and creating KWL charts or daily WILT reports. 

A final teaching emphasis that teachers see as congruent with the 
expectations of the retelling process is collaborative learning. Many 
teachers noticed, during the retelling process, that their students enjoyed 
retelling their stories to partners, but that they were not necessarily adept 
at helping each other to revise. To help students become better 
collaborative learners, some teachers report using "think-pair- share" 
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activities, peer conferencing, small group reporting and collaborative 
retellings of chapters in novels. 

Perhaps an illustration of one holistic lesson can best illustrate the 
kinds of literacy instruction that are happening at Delcroft and that 
teachers attribute, in part, to the effects of a congruent 
teaching/assessment plan. Carol Jones, a fifth grade teacher, recently 
integrated the study of Antarctica (Social Studies), weather (Science) and 
survival (Health) through a collaborative story writing activity. In 
heterogeneous groups of four, her students worked together to determine 
the setting, characters, problem and solution for their story. To aid them in 
their research, the Learning Support Teacher for the fifth grade led 
students in activities around various forms of note taking from listening 
and reading. Students also practiced sharing their notes and adding to or 
changing them-a study strategy as well as a form of revision. By the time 
students had finished their stories, they had engaged in an integrated 
listening/talking/reading/writing activity across the content areas. 

John Connor, another fifth grade teacher at Delcroft, observed that 
"With the thrust of writing across the curriculum, the written retelling 
project drives us to alter our teaching styles." (Questionnaire, 2/96) 
Comments like this and our observations indicate that the process has 
supported Delcroft teachers over the past three years as they have 
experimented with the kinds of holistic teaching and learning approaches 
which are advocated in the school's overall improvement plan. Although 
the actual retelling and rating sessions take place only twice a year, the 
project's consequences in terms of instructional change at the school have 
been widespread and powerful. It appears that the teaching/assessment 
project is an innovation which has started small and simple and which may, 
because of its congruence with the Delcroft's instructional goals, endure and 
continue to make a difference. 

Enhancing Professional Development and Collaboration 

In terms of professional development, teachers cite three major 
effects of the written retelling project: a wider, more informed perspective 
on the development of students’ writing abilities across the grade levels; 
increased knowledge of integrated instructional and assessment strategies; 
and a deepened respect for the value of colleagues' expertise and for the 
benefits of collaborative efforts. As one first grade teacher put it: "I have 
enjoyed my participation in this project. It has offered me guidance in my 
teaching, given me an opportunity to gauge each child's progress, and 
opened positive, helpful dialogue amongst my grade partners." (M.P. Juisti, 
Questionnaire 2/96) In addition to these teacher described effects, our 
observation is that, as Delcroft teachers have educated each other through 
their adaptation and implementation of the project, they have also gained 
investment in the process of reform more generally as well as confidence in 
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their abilities to question and to be a part of changing problematic or 
contradictory educational practices at their school. 

An unusual aspect of the written retelling project is that it involves 
teachers in looking, together, at the work of students who are not in their 
own classrooms. Delcroft teachers found that this process helped them 
learn the range of ability levels they might expect to see at their grade 
level and thus put where their students "are" in perspective. A fourth 
grade teacher commented that "the process has given me a wider, more 
global view of the range of my students' writings relative to the whole 
grade." (D. Coyle, Questionnaire 2/96). A first grade teacher, who has 
taught many of the students who have been labeled special education told 
us "I can see first hand that most children go through the same stages. 
Consequently, I usually feel less discouraged when I look at my students’ 
work." (P. Cascaden, 0,2/96). In addition to giving teachers a better sense 
of the writing abilities of the students at a particular grade level, 
participation in the rating sessions and in choosing retelling stories for their 
students has raised many questions for teachers about assessment 
strategies and writing development for students across grade levels. 
Teachers have begun to articulate concerns about needing to see an even 
wider picture in order to better understand how to best teach and assess 
writing to all students. 

According to many Delcroft teachers, the written retelling process 
has been a form of professional development because it has encouraged 
them to explore and to learn new strategies for the instruction and 
assessment of integrated language arts across the curriculum. The first 
step in any lasting professional development is an awareness of what 
students need; several teachers have reported that the retelling project has 
given them a sense of these needs and of how to meet them. Maureen 
Flicker, a learning Support teacher for the fourth grade, says that the 
retelling "gives you a sense of the importance of the writing process and 
keeps you in tune to the writing needs of your students. " Don Nickerson, 
another fourth grade teacher, writes that "Retelling increased my 
awareness of the importance of diversified strategies to meet the individual 
needs of my students" (Q.2/96). Toni Benson comments that, through the 
project, she has "learned new strategies for making children tune in to 
details and the order of events" (4th grade, 0,2/96). 

Along with instructional strategies, several Delcroft teachers mention 
learning new, related assessment strategies as an outcome of the retelling 
project. In particular, the cross-grade "assessment artifact share" session 
held the first year of the project (11/93) gave teachers a chance to discuss 
their assessment ideas and questions with each other. Teachers' 
(anonymous) reaction sheets from that meeting highlight their excitement 
at seeing "the many things \ye can assess and the many ways we can assess 
them." Some teachers also list specific questions the discussion raised for 
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them which they intend to explore, such as "how can we best assess a 
student's metacognitive abilities?" and "how do I explain to parents that 
invented spelling is okay?" Perhaps most interesting - in terms of the 
effect on teachers of their involvement in the retelling project - are the 
larger questions teachers raise about assessment practices at the school. 

For example, Carol Jones, a fifth grade teacher, feels that the retelling 
project has created a teaching situation in which "assessment drives 
instruction." However, she wonders why, then "are our report cards not 
reflective of these retelling assessments?" 

Another teacher wonders whether assessment/evaluation practices 
should be standardized by grade or by school, and what the implications of 
these different choices might be. A third asks "Don't we need time to live 
with the new literacy program and to develop assessment strategies more 
natural to our goals?" These questions are not easily answered, but that 
teachers are asking them reflects a thoughtful engagement in the larger 
issues of assessment and instruction that is one goal of any successful 
program of professional development. 

Professional development for teachers is central to the 
implementation of the Written Retelling Assessment Project. Rather than 
passively receiving information and ideas from outside experts, the Delcroft 
faculty has been actively involved in working together to shape a 
teaching/assessment approach that will work for their particular context. 
One of the most striking outcomes of this type of implementation process 
has been teachers' positive feelings about collaborating with their 
colleagues. Principal Bonnie George, at the end of the first year on the 
project, commented " I believe that the Retelling Project has provided our 
staff with opportunities for collaboration and collegiality and that these 
positive experiences have proven the power of working in a group toward a 
common goal." 

Her words are supported by the results of a teacher attitude survey 
completed by twenty three Delcroft teachers in June of 1994. The survey 
asked teachers to rate the influence of their participation in the project on 
five aspects of their professional lives. Possible ratings ranged from one 
(very negative effect) to five (very positive effect). The average rating for 
the item "How has your participation in the project affected your 
collaboration with fellow professionals?" was 4.87, an indication of a 
positive belief among teachers about the effect of the project on their 
collaboration with each other. More recent surveys and conversations with 
Delcroft teachers support his conclusion; their comments talk about the 
value of teacher dialogue and their growing sense that "teachers are the 
experts in what we do!" (Anon, reaction sheet). A fourth grade teacher 
compares the retelling project to other types of staff development in terms 
of its effects on collaboration "It has provided a much more intense 
opportunity for the teachers in the team to write and interchange ideas and 
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attitudes towards a specific educational area" (D. Coyle, Q.2/96). And a fifth 
grade teacher talks about the effects of this kind of 'interchange' on him- "I 
have developed an admiration and respect for my colleagues. Their 
expertise is invaluable to my success in the classroom. As a lifelong 
learner, my personal goals and expectations have changed" (J. Connor, Q. 
2/96). 

As they have developed a stronger sense of themselves as experts 
and as full participants in the processes of educational change at Delcroft, 
teachers have articulated what they see as their needs as professionals and 
change agents involved in educational reform. First and foremost among 
these needs is time (probably the most frequently used word on all of the 
reaction sheets we have collected to date). After the first all-faculty 
meeting we held to introduce the retelling project (5/93), teachers were 
already concerned about the need for time to properly implement the plan. 
Their reaction sheets listed, as concerns: meeting time for teachers, time to 
decide on anchor papers, time to meet regularly to evaluate writing 
throughout the year, and the time involved in assessing each paper. After 
the 11/93 assessment workshop, many teachers answered the question 
"What needs to happen in terms of professional development?" in similar, if 
more specifically informed ways: "We need more shared time with the 
primary team to develop a checklist that can be used at report card time"; 
"we need time to discuss strategies across grade levels"; "we need to slow 
down and have more of these kinds of reflective conversations." 

ANALYSIS OF THE EXPERIENCES OF THE FACULTY IN 
IMPLEMENTATION OF THE WRITTEN RETELLING PROCESS 

Our first research question was: How and how well did the teachers 
implement the Written Retelling reform effort? 

We used an adaptation of the Cole ( 1988) characteristics of 
assessment designed for instruction as a heuristic to answer this question 
based on the information reported in the previous section. 

Characteristic 1. Quality judged by effect on instruction. It 
is clear from our report that the written retelling process effected 
instruction in a number of ways. Children listened to, orally retold and 
wrote and read their versions of a whole narrative and expository piece. 
Teachers reported that they continued to use this holistic activity and a 
wide variety of other integrative language arts approaches in the course of 
the implementation of the written retelling process and over the years. 

The extent to which the teaching/leaming/assessment process was 
implemented and student learning was increased is evident from empirical 
evidence including teacher reports of the significance of their collaboration 
in the course of establishing and revisiting the rating session, teacher 
reports of significance of their classroom work with children over the first 
three years of the study, ana our own observational data as we observed 
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teachers at work with their peers in the rating sessions and in their 
classrooms. 

Characteristic 2. Design determined by instructional goals 
and locally scored. The instructional purpose of the written retelling 
process was to improve the quality of academic learning by having children 
construct meaning using integrated Hstening/talking/writing/reading to 
comprehend literary and expository texts. Teachers, working 
collaboratively at grade levels, assessed the written products of this 
learning process using procedures developed by Botel but adapted by the 
teachers at each grade level. 

Characteristic 3. Instructional raison d'etre. The written 
retelling assessment existed because it met an instructional purpose. Since 
it was believed that assessment has a powerful influence in driving 
instruction, the leadership and faculty chose an assessment process that 
was congruent with the district's Goals 2000 reform plan and with the 
Pennsylvania Framework for Reading, Writing and Talking across the 
Curriculum (Lytle and Botel, 1989). 

Characteristic 4. Teacher-mandated and adapted to local 
context. The retellings serving both as teaching/leaming and as 
assessment was presented for consideration as a framework. Because it 
was clearly a prototype learning experience that matched district goals, the 
principal and faculty of the Delcroft School volunteered to pilot the 
assessment framework process. As indicated earlier, teachers chose the 
literary and expository pieces and in a number of ways modified the 
framework for each grade. 

Characteristic 5. Test tasks of instructional value. Children's 
tasks in the retelling process from which the assessment information is 
derived are tasks that have instructional value in themselves. Various 
forms of the retelling process have long been used as holistic pedagogy 
with positive results in teaching reading comprehension, oral language 
development and apprehension of text structure. 

One of the limits of our assessment process is that retelling by itself 
does not tap into the many critical and creative kinds of experiences and 
opportunities for interpretation that should be part of a comprehensive 
democratic curriculum like reading and discussing whole books and stories, 
writing original material, writing about what stands out for them, etc. 

(Lytle and Botel, 1990, IRA/NCTE Standards for the Assessment of Reading 
and Writing, 1994.) This is why it is extremely important to have such a 
curriculum and to regard assessment as a continuous and comprehensive 
process involving many different kinds of assessments. As Supovitz (1997) 
concludes in his commentary, From Multiple Choice to Multiple Choices, "To 
seek greater equity, we have to develop a plethora of rigorously 
constructed assessment forties, understanding that-taken together, they 
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will be a fairer measure of that complex thing we call knowledge. A 
diverse society deserves a more diverse assessment system (p.37)." 

Characteristic 6. Provides immediate feedback, is informal 
and is used with other information over the course of the year. 
Since teachers gave, discussed and scored the assessment products (written 
retellings) in informal collaboration, they had rapid feedback which they 
used primarily as one means to chart the course for teaching/learning. The 
congruent processes of written retelling as both teaching/learning and 
assessment supported teachers' growing understanding of this linkage. 

ANALYSIS OF QUANTITATIVE OUTCOMES 

Our second research question was: Did the children make significant 
"value added" improvement in their content and mechanics scores on the 
narrative and expository written retellings? TABLES 1 and 2 present our 
findings. 



TABLE 1 

% OF TWO GROUPS OF CHILDREN WHO FROM BEGINNING 
KINDERGARTEN TO THE END OF FIRST GRADE SCORED A 7 OR 8 IN 
COMBINED NARRATIVE/EXPOSITORY RETELLINGS ON THE 
MODIFIED MARIE CLAY SCALE 



Date of retellings Grade 



GROUP 1 (N=68) 

September 1993 K 

June 1994 K 

September 1994 1 

June 1995 1 

GROUP 2 (N=71) 

September 1994 K 

June 1995 K 

September 1995 1 

June 1996 1 A 



% getting 7 or 8 

1 

4 

7 

82 

0 

4 

6 

79 
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At the beginning and end of both kindergarten and grade one, we 
used a teacher Modified Clay Scale to assess children's' writing- TABLE 1 
presents the per cent of two groups of children who scored 7 or 8 on that 
scale, each over a two year period. A 7 on the Clay Scale is defined as a 
recognizable story or exposition of three or more sentences and an 8 is 
defined a fairly well developed story or exposition. The children 
experienced a narrative and an expository retelling in September and June 
of each year and these scores were averaged. 

The following patterns are evident in TABLE 1: Overall, the two 
groups have practically identical patterns of scores in each of the four 
retelling experiences. As expected, at the beginning of Kindergarten, 
approximately 1% score 7 or 8; by the end of Kindergarten 4% do; by the 
beginning of first grade 7% do and by the end of first grade approximately 
80% do. In short, the great majority of children at Delcroft School over the 
two years have made a major leap into becoming "young authors". One of 
the limitations of the study is that we were not able to compare their 
performance with a control group to get a true "value added" result, but we 
believe from our consultation with Delcroft teachers and our own 
experiences that very few children in traditional programs would score at 
these levels at the end of first grade. 
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TABLE 2 

% OF 51 CHILDREN WHO WERE CONTINUOUSLY ENROLLED AT THE 
DELCROFT SCHOOL OVER A 5 YEAR PERIOD (9/93-6/98), SCORING 
MORE THAN 50% ABOVE THE NORM ON PRE AND POST NARRATIVE 
AND EXPOSITORY RETELLING ASSESSMENTS 



Content Mechanics 

Grade Narrative Expository Narrative Expository 



1 




see Table 


1 


2 


9/94 


6/94 


9/94 




0 


0 


0 


3 


9/95 


6/96 


9/95 




0 


14 


0 


4 


9/96 


6/97 


9/96 




37 


39 


25 


5 


9/97 


6/98 


9/97 




34 


36 


17 



6/95 


9/94 


6/95 


9/94 


6/95 


43 


0 


45 


0 


46 


6/96 


9/95 


6/96 


9/95 


6/96 


19 


0 


17 


0 


46 


6/97 


9/96 


6/97 


9/96 


6/97 


39 


25 


48 


39 


43 


6/98 


9/97 


6/98 


9/97 


6/98 


18 


20 


32 


20 


32 



TABLE 2 displays the % of the 51 children who were enrolled 
at Delcroft from beginning first grade to the end of the fifth grade who 
scored more than 50% above the local norm in four categories at the 
beginning and end of each school year. The percentages in TABLE 2 are 
often called "value added" (Olson, 1998) improvement, presumably a 
consequence of the instructional program and a reflection of actual gains in 
children’s achievement over the initial performance of the local norming 
group. A zero would suggest no measurable improvement over the local 
norm, while a higher per cent would suggest the extent of improvement 
over the local norm. * 
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There were a number of ways to display and analyze the Delcroft 
children's scores over the five years. The most detailed analysis would be 
to present in a table the per cent of children who, on the four variables, 
performed at the 1,2, 3, 4 and 5 levels. This would be most useful for 
teachers at each grade level. But for the purposes of this report we regard 
the structure of TABLE 2 as a useful prototype for answering our research 
question and for accountability purposes. 

The data in TABLE 2 suggest the following general effects of the 
retelling process over the five years on these 5 1 children: 1.) by reading 
across the rows it appears that significant improvement took place for the 
most part in all four categories from the beginning to the end of each school 
year, and 2.) by reading down the columns it appears that in the fourth 
year significant improvement took place at the beginning of the school year 
as well. This second pattern is particularly interesting, since it documents a 
quasi-experimental effect based on the fact that children's individual scores 
were always compared with the original norming for each grade. If 
students were the same as the norming (control) group who were not 
involved in the reform effort, 50% of them would be expected to score 
above the norm. If there were no gains in performance TABLE 2 would 
show a 0. As a matter of fact, 25 to 39% (over the four subassessments) of 
this group of 51 children (the "experimental group") at the beginning of the 
fourth grade scored above the original norm This second general finding 
holds up in the fifth grade and suggests that it may take a number years— 
three in the case of our study students— of commitment to a written 
retelling assessment process like ours to get some of the most significant 
improvement. 

A FINAL WORD 

With respect to qualitative assessment Cole provides us with this 
insight: "...it seems that we have concentrated far more effort to date on 
assessment designed for measurement than on assessment designed for 
instruction...However...there appears to be at least the possibility of new 
forms of assessment that will be directly used in the service of lea rnin g 
(1988, pp. 114,116)." In our report, we have used an adaptation of Coles' 
Characteristics of Assessment Designed for Instruction to analyze and 
document the very successful implementation process and outcomes of the 
reform effort at Delcroft. 

With respect to quantitative assessment, we agree with Hanushek and 
his colleagues when thy doubted that improved authentic assessment can 
lead reform by itself. As they have noted, "With accurate assessments of 
student performance, runs the argument, schools will automatically focus 
on improving test scores: teachers will adjust to improve student 
achievement, management excisions will become obvious, and reform will 
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occur almost spontaneously. We know of no evidence to suggest this is a 
realistic expectation (1994. p.126)." 

While our reform effort was centered around an authentic 
assessment, the design of our total effort was comprehensive with respect 
to the other necessary characteristics needed to insure success (Olson, 
1999), including: support of the administrators and faculty, the clarity, 
strength and flexibility of the design, the allocation of money and time for 
extensive and continuous professional development, 

the continuing on-site support of the design team and the appointment of a 
respected colleague from the staff to manage the reform process. 
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APPENDIX 

A FRAMEWORK FOR IMPLEMENTING WRITTEN RETELLINGS 
by Morton Botel, Wm. T. Carter Professor of Education and Child 

Development 

Graduate School of Education, University of Pennsylvania 

The WRITTEN RETELLING PROCESS will extend over three days for 
each of two activities: THE STORY TELLER/ AUTHOR ACTIVITY and 
THE NEWS RETELLING/NEWS REPORTER ACTIVITY 

In each of these activities, it helps children to think of themselves as 
taking on the roles of professional readers and writers (Seaver and 
Botel, 1991.) The activities are integrative learning experiences which 
involve the linking of reading, writing and talking in comprehending stories 
and informational texts. The assessment piece is the written outcome of a 
process involving all of the language arts. Thus, the assessment is 
congruent with teaching/learning. Does it provide us with a measure of 
comprehension, of ability to compose orally and in writing, of control of the 
conventions of oral and written language? Clearly, the answer seems to be 
yes to all of these developing abilities. 

1. THE STORY TELLER/ AUTHOR ACTIVITY 

The teacher begins this activity by engaging the class in a discussion 
of storytelling and authors. The idea here is that children should come to 
think of themselves as storytellers and authors. Furthermore, adopting 
goals such as reading/writing/talking like actors, lawyers, historians, and 
scientists provide socially significant, purposeful and motivational activities 
for students in the several content areas (Seaver and Botel 1991). 

A banner in the class might read: 

WE ARE LEARNING TO BE STORYTELLERS AND AUTHORS 

The teacher tells the class that one of the goals throughout the year is 
to practice storytelling and authoring to become more skillful in those art 
forms. She tells them that one of the many ways they will become more 
skillful will come from retelling and writing stories that she will read to 
them and that these stories will come from many different countries and 
cultures. That experience will help enrich and extend all students' 
knowledge of their own and others' ways of telling stories. Other activities 
which contribute to this craft include making up their own stories, telling 
and writing about stories they have read, telling and writing about stories 
they have heard and read from the point of view of one of the characters, 
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She also tells them that they will keep some of these written stories 
in folders called portfolios and that over the school year they will create a 
large collection of their writing and that she and they will be able to see 
how they have improved in their skills of reading and writing by 
examining their portfolios. 

Outline of the Story Retelling Process (to be adapted/refined by 
individual faculties at each grade level) 

Day 1. The teacher tells the class that she will read a short story 
aloud without stopping for discussion. A decision has to be made by all 
teachers at the same grade level regarding whether children will simply 
hear the story, have copies of the story while they listen, see it in a Big 
Book or on an overhead projector.) 

Following that, the teacher reads the story aloud again , during which 
and after which children will write down some brief notes or draw sketches 
of what stands out for them in the story. (The beginning and end of the 
year choices should be similar in terms of length and complexity. A short 
story about the length of an Aesop fable works well. In K and 1 we 
propose that teachers choose equivalent Big Books for the beginning and 
end of the school year experience.) 

Then, children pair up and take turns to retell their story to one 
another in their own words. They then write their first drafts. 

Kindergarten and first grade children are encouraged to draw and "write" 
their stories. 

The reason the teacher reads the story aloud is that it enables all 
children, even the lowest performing readers, to have access to the story 
and to participate fully in the activities at their own levels. 

Day 2. The next day the teacher encourages the children to revise or 
improve their first drafts by reading their drafts to their partners. 

Listening to themselves read their stories aloud has been found to 
stimulate revision. After the children read to each other, the teacher 
encourages partners to ask each other questions to get them to tell more 
about their stories. 

Then they do a second revision which involves re-examining their 
drafts by thinking about the following questions which are given to 
children as individual revision checklists as well as being posted on a 
permanent chart in the classroom. The teachers discuss these aspects of a 
well told story briefly with the children as they examine their stories. 

(They keep their drafts and staple them to the bottom of the revised 
versions.) 
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QUESTIONS I CONSIDER WHEN REVISING MY WORK 

check: Do I have all the characters, settings, actions and messages in 
the story? 

check: Do I want to change any of my words or sentences so they 
sound better? 



Day 3. The teacher gives the children a chance to edit their stories so 
they are ready for "publication." That is, they understand that they will 
complete a piece they regard as their best work— something they will be 
proud to share with classmates, parents and others. As part of the editing 
step, students should refer to the questions on the chart below. 

QUESTIONS I CONSIDER WHEN EDITING MY WORK 
check: Are my spellings correct? 
check: Are my capitalizations correct? 
check: Is my punctuation correct? 



Both of these sets of questions can be modified by the teachers at 
each grade level to accommodate children’s' developmental levels and local 
expectations. As before these are also provided as individual editing 
checklists to each child. 

THE NEWS RETELLING/NEWS REPORTER ACTIVITY 
A similar set of teaching/learning and assessment procedures 
can be used with informational writing. In this case children are taught to 
think of themselves as news reporters. The chart posted in the room for the 
Day 2 revision activity is like the one for story retelling except for the first 
check: 



QUESTIONS I CONSIDER WHEN REVISING MY WORK 
check: Do I include the who, what, where, when, why in my 
retelling? 

check: Do I want to change any of my words or sentences so they 
sound better? 



The classroom banner for this activity says 

WE ARE LEARNING TO BECOME NEWS REPORTERS 

A suggested source for the pieces to be read aloud would be the cover 
story found in The Weekly Reader, Scholastic periodicals, etc. (Other 
activities over the year would involve children writing pieces for a class 
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newspaper on current events of interest, writing about a historical event as 
if they were present as reporters, etc.) 

Now, let's analyze what we have done so far. When children have 
completed these activities they have (1) practiced the storytelling and 
author's craft, (2) practiced related comprehension skills involving story 
structure relationships among characters, setting, actions, story problems 
and resolutions, (3) been engaged in very integrative language experiences 
where they have listened/ talked/written/read, (4) collaborated with peers, 
(5) worked through a writing process and (6) been engaged in a productive 
form of assessment as they engaged in self reflection through the revision 
process. 

Qualitative Assessment 

There are three kinds of qualitative assessments involved in 
implementing this framework. First, in teaching this way teachers are able 
to observe the children as they participate in the activity. While doing that 
they are, in fact, assessing children's writing processes and products to 
determine the specific needs of individual children and general needs of 
the class. Such qualitative assessment is one of the most helpful forms of 
assessment since it so powerfully informs teaching. 

Second, later in this framework it will be seen that teachers at grade 
level look together at the children's written work to rank them. While 
they do so they are, in fact, looking closely at the qualities of children's 
writing where they can make instructional as well as ranking decisions. 
Teachers' shared discussion of children's work might be likened to a 
staffing conference of doctors as they examine the patient's records as the 
basis for decisions on further treatment. In the case of teachers, they will 
gain greater insight into the needs of their children. 

Third, there should be a continuous study of the implementation 
process involving observation and reporting of the collaborative assessment 
process, classroom teaching/leaming activities, interviews with teacher on 
what changes they are making, interviews with children and examination 
of the portfolios children are keeping. 

Quantitative Assessment 

There are stakeholders who expect to have the results of more 
quantitative assessment. These always include administrators, board 
members, parents, and other taxpayers. Whereas the teacher and children 
are more interested in the process of personal reflection about 
teaching/learning the other stakeholders usually ask for evidence of 
outcomes or accountability in quantitative terms. They want to know 
whether children are making significant and measurable gains in their 
performance. Failure to produce such evidence often undermines the 
confidence and support of these stakeholders. 
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To provide for this need I propose two ways (one for grades K and 1, 
and one for grades 2 and above) for rating the story retelling piece and the 
informational piece described earlier to be given at the beginning and end 
of each school year. The differences between the beginning and end of the 
year scores, particularly over a period of years, will document growth in 
the performance of groups of children and individual children as well as 
provide helpful evidence of the effectiveness of the curriculum. 



Rating Children’s Writing in Kindergarten and Grade 1* 

K and 1 teachers collaborate in rating children's writing using this 
scale or one adapted from it by the teachers in each school. 



1 Drawing only 

2 Scribble writing 

3 Letters only 



4 Any recognizable word 
related to the story 



5 Any recognizable phrase related 

to the story 

6 Any recognizable sentence related 

to the story 

7 Any recognizable story of 

three or more sentences 
related to the story 

8 A fairly well developed story 

with characters and actions 



*adapted from Marie Clay, Early Detection of Reading Difficulties . 1979. 
Heinemann, and further modified by the Delcroft K-l teachers. 

Rating Children's Writing in Grades 2 and Above 
Ratings in grade 2 and above should be done by all the teachers in 
each grade level at each school. This will assure ratings that are school 
based at the start, i.e. based on the norm of that school so that it will be 
possible to compare future performance of children in that school with the 
norm of their own school. (This is like "handicapping" in sports and contests 
where less expert players get advantages to equalize the chances of 
winning. People speak of this as "providing a level playing field.") 

All the teachers of the same grade level meet and combine the 
revised writing pieces of all children in their classes. (All papers should 
have had the teachers’ and children's names and the date on the back of 
the piece.) Teachers divide t^he papers into four equal piles in terms of the 
quality of the content only in the first round. (A second round will focus 
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on the conventions of writing only, thereby providing another score for 
each piece of writing.) The top pile will include the best quarter of the 
papers which will be marked C4(C is for content), the second top fourth are 
each marked C3, the third fourth are each marked C2, and the bottom 
fourth are each marked Cl. These should be recorded on the back of each 
piece. 

In the process of rating the pieces, teachers, by the nature of the 
collaboration, will have negotiated agreement. In making the judgment, the 
pieces are read aloud by one of the teachers (taking turns) with the other 
teachers listening for what the children were trying to express. This 
negotiated process (while looking for qualities such as organization, 
accuracy, choice of words, originality, insights, and humor) are likely to 
contribute to high agreement (reliability) of teacher judgments. Again, the 
conventions of writing should be ignored in this round. (Occasionally a fine 
piece of writing will not be an accurate rendition of the original story, but 
may have a creative flair. It is best to keep open to unexpected 
possibilities. After all, one of the main goals of the retelling experience is 
helping students become better storytellers.) 

Teachers then choose the lowest pieces in quartiles 2, 3, and 4 and 
the strongest paper in quartile 4 designating these four pieces as the 
anchor papers for content ratings. Schematically (below) the display of 
papers are represented with the right hand X representing the best paper 
and the left hand x representing the weakest paper. The capital Xs at the 
left of the 2s, 3 s, and 4s and the one at the right of the 4s are the Anchor 
Papers. They become the yardstick for the rating of future papers. Thus, 
the question teachers ask in subsequent rating sessions as they look at 
each paper at the end of the year and at the beginning and end of later 
years is this: 

Where would this paper have been placed in the original 
ranking? 

In future assessments, If the piece is as good as or better than the 4 
Anchor Paper, it would receive a 4 rating. If the paper is as good as or 
better than the 3 Anchor Paper, but not as good as the 4 Anchor Paper, it 
would receive a 3 rating. If the paper is as good as or better than the 2 
Anchor Paper, but not as good as the 3 Anchor Paper, it would receive a 2 
rating. There is one additional question that can be asked in later ratings. 

Is the written retelling as good as or better than the best paper 
in the original rating where anchor papers were determined? Such 
papers could be given a rating of 5. This means that children who got a 4 
rating initially and who have improved enough can get a 5. 
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Is 2s 3s 4s 

xxxxxxxxxxxxxx Xxxxxxxxxxxxx Xxxxxxxxxxxxxxx XxxxxxxxxxxxxxX 



C2 


C3 


C4 


C5 


M2 


M3 


M4 


M5 



These five pieces (Xs/Anchor Papers) should be marked with a capital C2, 
C3, C4 and C5 (C is for content) on the back of the pieces and these writing 
pieces should be Xeroxed and the originals put back in the pile. 

Next, teachers rank all the papers again from high to low but this 
time based entirely on the conventions of writing. Unlike the rating for 
content where they simply listen to the stories, in this reading all teachers 
have to look at the pieces to make their judgment based on spelling, 
punctuation, capitalization and usage. This time the anchor pieces in each 
quartile are marked M2 (M is for mechanics), M3, M4 and M5. As in the 
case of the content rating, the lowest paper in M2, M3, and M4 and the 
highest paper in M4 are chosen as Anchor Papers and marked on the back 
of the pieces. These are Xeroxed and the originals returned to the pile. As 
with the Content rating, subsequent papers at the end and beginning of the 
school year are rated by reference to these anchor papers. 

Use of the Anchor Papers 

By consulting these Anchor papers from time to time over the year, 
teachers will be able to learn more about the instructional needs of their 
children. 

All of the agreements of teachers at any grade level regarding the 
conduct of the beginning teaching/leaming/assessment process become 
standardized for that grade in the school for the several years of the study. 
The agreements along with the student anchor papers become the standard 
procedures for the beginning and end of the year's 
teaching/learning/assessment activities. 

Of course, this sets no limit on the variations and richness of learning 
activities teachers use throughout the year. But if the quantitative results 
are to have any meaning there needs to be an agreed upon and fairly 
invariable process for the first and later conduct of the written retelling 
experiences. 
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